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What is claimed is: 

1 . A method of determining a revision of a test suite of a model-based 
diagnostic testing system comprising: 

evaluating a diagnostic efficacy of the test suite using a probability of a 
diagnosis. 

2 . The method of Claim 1 , wherein the evaluation comprises : 
suggesting a test to add to the test suite to adjust an overall test coverage of the 

test suite. 

3 . The method of Claim 2, wherein suggesting a test comprises: 
creating a simulation database of the test suite; 

determining a probability of a correct diagnosis and a probability of an incorrect 
diagnosis for the test suite using the database; and 

creating a list of suggested tests from the determined probabilities. 

4. The method of Claim 3, wherein each suggested test on the list comprises 
a test coverage. 

5 . The method of Claim 3 , wherein the evaluation further comprises : 
identifying a test to delete from the test suite that comprises: 
determining a probability of a correct diagnosis for a modified test suite using 

the database, the modified test suite having a selected test removed from the test suite; 

computing an efficacy value associated with the selected test using the 
determined probabilities of a correct diagnosis for the test suite and the modified test 
suite; and 

generating a list of deletable tests and associated efficacy values. 

6. The method of Claim 1 , wherein the evaluation comprises: 

identifying a test to delete from the test suite, the deletable test having a minimal 
effect on an overall diagnostic efficacy of the test suite. 

7. The method of Claim 6, wherein identifying a test comprises: 



-26- 



PDNO 10980101-1 



creating a simulation database of the test suite; 

determining a probability of a correct diagnosis for the test suite using the 
database; 

determining a probability of a correct diagnosis for a modified test suite using 
5 the database, wherein the modified test suite is the test suite having a selected test 
removed; 

computing an efficacy value for the modified test suite using the determined 
probabilities; and 

generating a list of deletable tests using the computed efficacy values. 

8. The method of Claim 7, wherein determining a probability for a modified 
test suite is repeated for a plurality of modified test suites, each modified test suite of 
the plurality being the test suite having a different selected test removed. 

9. The method of Claim 8, wherein the selected test associated with the 
modified test suite having a low computed efficacy value relative to other modified 
test suites is the deletable test. 

10. The method of Claim 7, wherein evaluating further comprises: 
suggesting a test to add to the test suite to adjust an overall test coverage of the 

test suite that comprises: 

determining a probability of an incorrect diagnosis for the test suite using the 
20 database; and 

creating a list of tests to add from the determined correct and incorrect 
probabilities for the test suite. 

11. A method of evaluating a diagnostic efficacy of a test suite of a model- 
based diagnostic testing system comprising: 

25 creating a simulation database of the test suite; 

determining a probability of one or both of a correct diagnosis and an incorrect 
diagnosis for the test suite using the database; and 

using the determined probability to evaluate the test suite. 
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12. The method of Claim 11, wherein creating a simulation database 
comprises: 

simulating an application of the test suite to a device under test, the device under 
test comprising one or more components; and 

recording a probable result of the application in the simulation database, the 
simulation database being represented by a table having a plurality of columns and a 
plurality of rows, the plurality of columns comprising a component pattern, a test 
result pattern, and a number of occurrences, 

wherein the component pattern encodes which component is good or bad, each 
component of the device under test being represented by a unique position number 
within the component pattern, 

wherein the test result pattern encodes which of the tests of the test suite failed 
or passed, each test in the test suite being represented by a unique position within the 
test result pattern, 

wherein the number of occurrences represents a number of times that a given 
combination of the component pattern and the test result pattern occurred during a 
simulation, the number of occurrences being an integer greater than or equal to zero, 
and 

wherein each row of the plurality of rows corresponds to a different unique 
pattern of good and bad components. 

1 3 . The method of Claim 1 2, wherein determining a probability of one or both 
of a correct diagnosis and an incorrect diagnosis comprises: 

copying to a database copy only those rows of the created simulation database 
with only one bad component in the component pattern column, all other components 
in the respective row being good; 

sorting the database copy based on the test pattern column, such that the rows 
with a given test pattern are adjacent to one another, the adjacent rows forming a 
group of rows; 

examining each group of rows to locate a row within each group having a 
largest number of occurrences relative to other rows within the respective group; and 

assigning a diagnosis d to each group, the diagnosis d being the position number 
of the bad component for the located row. 
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14. The method of Claim 13, wherein determining a probability of one or both 
of a correct diagnosis and an incorrect diagnosis further comprises creating and 
initializing a matrix M such that matrix elements M(i,j) of the matrix M are equal to 
zero for all i and j, where i is an integer that ranges from one to m+l and where j is an 
integer that ranges from one to m, where m is the number of tests in the test suite. 

15. The method of Claim 14, wherein for each group having a test pattern that 
represents no failed tests, determining a probability further comprises: 

adding iteratively for each row r in the group the number of occurrences value 
of the row r to a current value of the matrix element M(m+\, b) to generate a next 
value of the matrix element M(m+1 , b), where b is a position number of the bad 
component for the row r. 

16. The method of Claim 15, wherein for each group having a test pattern that 
represents at least one failed test, determining a probability further comprises: 

adding iteratively for each row r in the group the number of occurrences value 
of the row r to a current value of the matrix element M(d, b) to generate a next value 
of the matrix element M(d, b). 

17. The method of Claim 16, wherein the determined probability of a correct 
diagnosis P cor r is calculated using 

P corr = Z/E 

where £ is a sum of diagonal elements M(i, i) of the matrix M for i equals one to 
m and E is a sum of all number of occurrence values in the database copy. 

18. The method of Claim 1 7, wherein using the determined probability to 
evaluate the test suite comprises: 

suggesting a test to add to the test suite to improve diagnostic efficacy. 

19. The method of Claim 18, wherein suggesting a test comprises: 

finding a relatively largest value element M(i,j) in the matrix M, where i is not 
equal to j, the element M(i,j) representing the probability of incorrectly diagnosing 
component i as the bad component when component j is actually bad; and 
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suggesting a test t having high coverage for component i and one of either low 
coverage for component j, if j is not equal to m+l, or coverage being irrelevant, if/' is 
equal to m+l . 

20. The method of Claim 19, wherein suggesting a test further comprises 
creating a list of suggested tests, wherein creating a list comprises: 

repeating finding and suggesting for each element of a set of largest value 
elements M(i,j), a test being suggested for each element of the set of elements M(iJ); 
and 

computing a score for each of the suggested tests, the score being computed by 
dividing the element value M(i,j) by the total accumulated number of occurrences E. 

21 . The method of Claim 20, wherein the list of suggested tests is represented 
in one or both of human readable form or machine-readable form. 

22. The method of Claim 1 7, wherein using the determined probability to 
evaluate the test suite comprises: 

identifying a test t of the test suite that may be deleted from the test suite. 

23 . The method of Claim 22, wherein for identifying a test t to delete from the 
test suite, the method further comprises: 

determining a probability of a correct diagnosis P CO rr,t for a modified test suite 
using the database, the modified test suite having a selected test t removed from the 
test suite; 

computing an efficacy value for the modified test suite using the determined 
probabilities for the test suite and for the modified test suite; and 

generating a list of deletable tests, the deletable tests having a lowest associated 
efficacy relative to efficacies of other tests in the test suite. 

24. The method of Claim 23, wherein determining a probability of a correct 
diagnosis P CO rr,t for a modified test suite associated with each of the tests t in the test 
suite comprises using a modified database created from the database copy, wherein 
the modified database is created comprising: 

copying the database copy into another database copy; 
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selecting a test t to remove from the test suite; 

deleting a position from each test pattern associated with the selected test t from 
the other database copy; and 

copying rows of the other database copy into a modified database, such that any 
rows that have identical values for the component pattern and the test pattern are 
combined together in the modified database, 

wherein in each row of the modified database that represents a set of combined 
rows from the other database copy the number of occurrences is a sum of the number 
of occurrence values for the combined rows. 

25. The method of Claim 24, wherein the probability of a correct diagnosis 
Pcorr.t for each of the modified test suites is determined in a manner analogous to 
determining the probability of a correct diagnosis P corr for the test suite. 

26. The method of Claim 24, wherein determining a probability of a correct 
diagnosis P CO rr,t for the modified test suite T using the modified database comprises: 

summing a largest number of occurrences value v max found for each unique test 
pattern value within the modified database; and 

dividing the v max sum by a total number of occurrences E t , where the total 
number of occurrences E t is the sum of all numbers of occurrences in the modified 
database. 

27. The method of Claim 23, wherein computing an efficacy value for each of 
the tests in the test suite comprises computing a difference between the determined 
probability of a correct diagnosis P CO rr,t for the modified test suite corresponding to a 
selected test t and the determined probability of a correct diagnosis P corr for the test 
suite. 

28. The method of Claim 27, wherein the computed efficacy e(t) value further 
comprises a cost metric c{t) associated with the test t, where s(t) = c(t)-(P corrJ - P CO rr)- 

29. The method of Claim 27, wherein the generated list of deletable tests 
comprises an associated efficacy value for each of the deletable tests. 
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30. The method of Claim 23, wherein the generated list is represented in one 
or both of human readable form and in machine-readable form. 

3 1 . The method of Claim 1 1 , wherein the created simulation database 
comprises a Monte Carol simulation of the device under test model, the database 
having a set of entries, each entry having a field for a number-of-occurrences value, a 
field for a test result pattern, and a field for a component state pattern. 

32. A system that determines efficacy of a test suite of a model-based 
diagnostic testing system comprising: 

a processor; 
a memory; and 

a computer program stored in the memory and executed by the processor, 
wherein the computer program comprises instructions that, when executed by the 
processor, implement evaluating the test suite using a probability of a diagnosis to 
determine the efficacy. 

33 . The system of Claim 32, wherein the instructions that evaluate the test 
suite comprise one or both of suggesting a test to add to the test suite, and identifying 
a test to delete from the test suite. 

34. The system of Claim 32, wherein the instructions that evaluate the test 
suite comprise: 

creating a simulation database of the test suite; 

determining a probability of one or both of a correct diagnosis and an incorrect 
diagnosis using the database; and 

using the determined probability to evaluate the test suite. 

35. The system of Claim 34, wherein using the determined probability of both 
a correct diagnosis and an incorrect diagnosis comprises creating a list of suggested 
tests to add to the test suite, each suggested test having an associated test coverage. 

36. The system of Claim 34, wherein the instructions that evaluate the test 
suite further comprise: 
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determining a probability of a correct diagnosis for a modified test suite using 
the database, the modified test suite having a selected test removed from the test suite; 
and wherein using the determined probability comprises: 
computing an efficacy value for the modified test suite using the determined 
5 probability of a correct diagnosis for both the test suite and the modified test suite; 
and 

generating a list of tests to delete from the test suite based on the computed 
efficacy value. 

37. The system of Claim 36, wherein determining a probability of a correct 
10 diagnosis for a modified test suite is repeated for different modified test suites, each 

different modified test suite having an associated different selected test being 
removed. 

38. The system of Claim 33, wherein suggesting a test to add to the test suite 
and identifying a test to delete from the test suite each comprise a list of respective 
tests, the lists being represented in one or both of human readable form or machine- 
readable form. 
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